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Problem of the Day 


The knapsack problem is as follows: given a set of integers 
S = {si, 52 ,..., s n }, and a given target number T, find a 
subset of S which adds up exactly to T. For example, within 
S = {1, 2, 5, 9, 10} there is a subset which adds up to T = 22 
but not T = 23. 

Find counterexamples to each of the following algorithms for 
the knapsack problem. That is, give an S and T such that 
the subset is selected using the algorithm does not leave the 
knapsack completely full, even though such a solution exists. 



Solution 


• Put the elements of S in the knapsack in left to right order 
if they fit, i.e. the first-fit algorithm? 

• Put the elements of S in the knapsack from smallest to 
largest, i.e. the best-fit algorithm? 

• Put the elements of S in the knapsack from largest to 
smallest? 



The RAM Model of Computation 


Algorithms are an important and durable part of computer 
science because they can be studied in a machine/language 
independent way. 

This is because we use the RAM model of computation for 
all our analysis. 

• Each “simple” operation (+, =, if, call) takes 1 step. 

• Loops and subroutine calls are not simple operations. 
They depend upon the size of the data and the contents 
of a subroutine. “Sort” is not a single step operation. 



• Each memory access takes exactly 1 step. 

We measure the run time of an algorithm by counting the 
number of steps, where: 

This model is useful and accurate in the same sense as the 
flat-earth model (which is useful)! 


Worst-Case Complexity 


The worst case complexity of an algorithm is the function 
defined by the maximum number of steps taken on any 
instance of size n. 



Problem Size 







Best-Case and Average-Case Complexity 


The best case complexity of an algorithm is the function 
defined by the minimum number of steps taken on any 
instance of size n. 

The average-case complexity of the algorithm is the function 
defined by an average number of steps taken on any instance 
of size n. 

Each of these complexities defines a numerical function: time 
vs. size! 



Exact Analysis is Hard! 


Best, worst, and average are difficult to deal with precisely 
because the details are very complicated: 



It easier to talk about upper and lower bounds of the function. 
Asymptotic notation (0,0, Q) are as well as we can 
practically deal with complexity functions. 






Names of Bounding Functions 


• g(n) = 0(f(n)) means C x f{n) is an upper bound on 

dip)- 

• g(n) = Q,(f(n)) means C x f(n) is a lower bound on g(n). 

• g(n) = 0(/(n)) means Ci x /(n) is an upper bound on 
g(n) and C 2 x /(n) is a lower bound on g{n). 

C , Ci, and C 2 are all constants independent of n. 



O, $ 7 , and 0 



(a) 


(b) 


(c) 


The definitions imply a constant no beyond which they are 
satisfied. We do not care about small values of n. 











Formal Definitions 


• f(ri) = 0(g(n)) if there are positive constants no and c 
such that to the right of no, the value of f(n) always lies 
on or below c • g(n). 

• /(n) = Q(g(n)) if there are positive constants no and c 
such that to the right of no, the value of /(n) always lies 
on or above c • g(n). 

• /(n) = 0(g(n)) if there exist positive constants no, ci, and 
C 2 such that to the right of no, the value of /(n) always lies 
between c\ • g{n) and C 2 • g(n) inclusive. 



Big Oh Examples 


3 n 2 
3?t 2 
3n 2 


100 n + 6 = 0(n 2 ) because 3 n 2 > 3n 2 — lOOn + 6 
100n + 6 = 0(n 3 ) because .01 n 3 > 3n 2 — 100 n + 6 
lOOn + 6 ^ 0(n ) because c - n < 3 n 2 when n > c 


Think of the equality as meaning in the set of functions. 



Big Omega Examples 


3 n 2 — 100 n + 6 = H(n 2 ) because 2.99 n 2 < 3 n 2 — 100n + 6 
3 n 2 — 100 n + 6 ^ H(n 3 ) because 3n 2 — 100 n + 6 < r? 

3 n 2 — lOOn + 6 = Q(n) because 10 10 °n < 3 n 2 — 100 + 6 



Big Theta Examples 


3n 2 — lOOn + 6 = 0(n 2 ) because O and 0 
3n — 100 n + 6 ^ 0(n ) because O only 
3n 2 — 100 n + 6 ^ G(n) because Q only 



Big Oh Addition/Subtraction 


Suppose f(n) = 0(n 2 ) and g(ri) = 0(n 2 

• What do we know about g'(n) = f{n) + g{n)l Adding the 
bounding constants shows g\n) = 0(n 2 ). 

• What do we know about g"{n) = f(n) — \g{n)\l Since 
the bounding constants don’t necessary cancel, g"(n) = 
0(n 2 


We know nothing about the lower bounds on g' and g" 
because we know nothing about lower bounds on / and g. 




Big Oh Multiplication by Constant 


Multiplication by a constant does not change the asymptotics: 





Big Oh Multiplication by Function 


But when both functions in a product are increasing, both are 
important: 


0{f(n)) * 0(g(n)) -»• 0{f(n) * g{n)) 

Q(/(n)) * Q{g(n)) -+ Q(f{n) * g(nj) 
0(/(n)) * 6{g{n)) 0(/(n) * fif(n)) 





